Method and apparatus for creating augmented reality content

ABSTRACT

Disclosed herein is a method for creating augmented reality content. The method for creating augmented reality content includes: extracting a target area, to which augmented reality content will be applied, from an object included in an image; providing a template for the creation of the augmented reality content that will be applied to the extracted target area; when a user inputs user content via the template, creating the augmented reality content by using the user content and the metadata of the target area; and performing rendering so that the created augmented reality content is applied to the target area of the image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of Korean Patent Application No.10-2019-0149917 filed on Nov. 20, 2019, which is hereby incorporated byreference herein in its entirety.

BACKGROUND 1. Technical Field

Embodiments disclosed herein relate generally to a method and apparatusfor creating augmented reality content, and more specifically to amethod and apparatus that enable a user to directly create augmentedreality content intended to be applied to an object included in animage.

This work was supported by Institute of Information & CommunicationsTechnology Planning & Evaluation (IITP) grant funded by the Koreagovernment (MSIT)(No. 2017-0-01445-003, The Development of Deep Learningbased Mobile Markerless AR App for Smart IOT).

2. Description of the Related Art

Recently, the number of people using social media has been graduallyincreasing. Live internet broadcasting, which is a representative formof social media, is different from media such as TV broadcasting in thata general user can be a broadcaster and a broadcaster and a viewer cancommunicate with each other. For example, a viewer can deliver his orher opinion or requirements through a chat window while viewing a livebroadcast, and can express his or her support by donating an itemcorresponding to money to a broadcaster.

In order to induce users to more actively participate in social media,various communication methods need to be provided. Therefore, there isan increasing need for the development of related technology.

Meanwhile, the above-described background technology corresponds totechnical information that has been possessed by the present inventor inorder to contrive the present invention or which has been acquired inthe process of contriving the present invention, and can not necessarilybe regarded as well-known technology which had been known to the publicprior to the filing of the present invention.

SUMMARY

Embodiments disclosed herein are directed to a method and apparatus thatenable a user viewing an image to directly create augmented realitycontent intended to be applied to an object included in an image.

According to an aspect of the present invention, there is provided amethod for creating augmented reality content, the method including:extracting a target area, to which augmented reality content will beapplied, from an object included in an image; providing a template forthe creation of the augmented reality content that will be applied tothe extracted target area; when a user inputs user content via thetemplate, creating the augmented reality content by using the usercontent and the metadata of the target area; and performing rendering sothat the created augmented reality content is applied to the target areaof the image.

According to another aspect of the present invention, there is provideda non-transitory computer-readable storage medium having stored thereona program that performs a method for creating augmented reality content,wherein the method for creating augmented reality content includes:extracting a target area, to which augmented reality content will beapplied, from an object included in an image; providing a template forthe creation of the augmented reality content that will be applied tothe extracted target area; when a user inputs user content via thetemplate, creating the augmented reality content by using the usercontent and the metadata of the target area; and performing rendering sothat the created augmented reality content is applied to the target areaof the image.

According to another aspect of the present invention, there is provideda computer program that is executed by an apparatus for creatingaugmented reality content and is stored in a storage medium to perform amethod for creating augmented reality content, wherein the method forcreating augmented reality content includes: extracting a target area,to which augmented reality content will be applied, from an objectincluded in an image; providing a template for the creation of theaugmented reality content that will be applied to the extracted targetarea; when a user inputs user content via the template, creating theaugmented reality content by using the user content and the metadata ofthe target area; and performing rendering so that the created augmentedreality content is applied to the target area of the image.

According to still another aspect of the present invention, there isprovided an apparatus for creating augmented reality content, theapparatus including: a communication interface configured to communicatewith an external device in order to transmit and receive data to andfrom the external device; a memory configured to store a program for thecreation of augmented reality content; and a controller configured tocreate augmented reality content by executing the program; wherein thecontroller is further configured to extract a target area, to which theaugmented reality content will be applied, from an object included in animage received via the communication interface, to provide a templatefor creation of the augmented reality content, to be applied to theextracted target area, to a user terminal via the communicationinterface, to, when a user inputs user content via the template, createthe augmented reality content by using the user content and the metadataof the target area, and to perform rendering so that the createdaugmented reality content is applied to the target area of the image.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features, and advantages of the presentinvention will be more clearly understood from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a view showing a social media system that supports a methodfor creating augmented reality content according to an embodiment;

FIG. 2 is a block diagram illustrating the configuration of an apparatusfor creating augmented reality content according to an embodiment;

FIGS. 3 to 6 are flowcharts illustrating methods for creating contentaccording to embodiments;

FIG. 7 shows screens that are displayed on a user terminal at the stepof providing a template for the creation of content to a user in amethod for creating augmented reality content according to anembodiment; and

FIG. 8 is a view showing an example in which augmented reality contentcreated by a method for creating augmented reality content according toan embodiment is applied to a live broadcast image.

DETAILED DESCRIPTION

Various embodiments will be described in detail below with reference tothe accompanying drawings. The following embodiments may be modified tovarious different forms and then practiced. In order to more clearlyillustrate the features of the embodiments, detailed descriptions ofitems that are well known to those having ordinary skill in the art tothe following embodiments pertain will be omitted. In the drawings,portions unrelated to the following description will be omitted.Throughout the specification, similar reference symbols will be assignedto similar portions.

Throughout the specification and the claims, when one component isdescribed as being “connected” to another component, the one componentmay be “directly connected” to the other component or “electricallyconnected” to the other component through a third component.Furthermore, when any portion is described as including any component,this does not mean that the portion does not exclude another componentbut means that the portion may further include another component, unlessexplicitly described to the contrary.

FIG. 1 is a view showing a social media system that supports a methodfor creating augmented reality content according to an embodiment.Referring to FIG. 1, the social media system according to the presentembodiment may include a camera 10, a server 100, and a user terminal20.

The camera 10 transmits a photographed image of a broadcaster 1 to theserver 100.

The server 100 receives the photographed image from the camera 10 andthen transmits the photographed image to the user terminal 20, therebyproviding a live broadcast service. Furthermore, the server 100 mayprovide support so that augmented reality content directly created by auser 2 is applied to an image that is broadcast in real time. In otherwords, the server 100 may practice the method for creating augmentedreality content according to the present embodiment. Although theembodiment in which the server 100 performs two roles, i.e., theprovision of a live broadcast service and the creation of augmentedreality content, is illustrated in FIG. 1, two separate servers mayperform the provision of a live broadcast service and the creation ofaugmented reality content, respectively.

A detailed description of the server 100 performing a process for thecreation of augmented reality content will be given later.

The user terminal 20 may display the image received from the server 100on a screen, and the user 2 may view a live broadcast via the userterminal 20. Furthermore, the user terminal 20 may provide a userinterface (UI) for the creation of augmented reality content to the user2, and the user 2 may directly create desired augmented reality contentvia the UI and apply the augmented reality content to an image that isbroadcast in real time.

The user terminal 20 may directly perform at least part of a processincluded in the method for creating augmented reality content. However,in the present embodiment, it is assumed that the server 100 performsthe overall process of the method for creating augmented reality contentand the user terminal 20 simply displays the UI for the creation ofaugmented reality content on a screen via communication with the server100.

The user terminal 20 may be implemented using a computing device capableof performing image output, such as a desktop, a notebook, a smartphone,or the like. The user 2 may make a request for the creation of augmentedreality content to be applied to the image at a desired point in timewhile viewing the image in real time via the user terminal 20, and theuser terminal 20 may transmit the request to the server 100, therebyallowing a process for the creation of the augmented reality content tobe initiated.

Meanwhile, although the case in which the server 100 supports thecreation of augmented reality content in the social media system inwhich an image is broadcast in real time via the server 100 isillustrated in FIG. 1 as an example, a computing device capable ofdisplaying an image, such as the user terminal 20, may support thecreation of augmented reality content in anther embodiment, in whichcase the user terminal 20 may also support the creation of augmentedreality content to be applied to an image stored in the user terminal 20other than an image that is received from the outside and played back inreal time.

FIG. 2 is a block diagram illustrating the configuration of an apparatusfor creating augmented reality content. The apparatus for creatingaugmented reality content, which is shown in FIG. 2, corresponds to theserver 100 of FIG. 1. As described above, the server 100 according tothe present embodiment provides a live broadcast service and,simultaneously, performs a process for the creation of augmented realitycontent.

Referring to FIG. 2, the server 100 according to the present embodimentmay include a communication interface 110, a controller 120, and amemory 130.

The communication interface 110 is a component configured to transmitand receive data to and from an external device, and may supportwired/wireless communication using various methods. For example, thecommunication interface 110 may receive a photographed image of thebroadcaster 1 from the camera 10. Furthermore, the communicationinterface 110 may transmit the image received from the camera 10 to theuser terminal 20, and may transmit and receive data to and from the userterminal 20 in order to perform a process required for the creation ofaugmented reality content. For this purpose, the communication interface110 may be implemented using a communication chipset configured tosupport various communication protocols.

The controller 120 is a component including at least one processor, suchas a central processing unit (CPU), and controls the overall operationof the server 100. The controller 120 may provide a live broadcastservice by executing a program previously stored in the memory 130, andmay perform a process for the creation of augmented reality content tobe applied to an image that is broadcast in real time.

In other words, the controller 120 may transmit the image received fromthe camera 10 to the user terminal 20 in real time, may provide a toolfor the creation of augmented reality content to the user 2 via the userterminal 20, and may create augmented reality content and apply theaugmented reality content to an image broadcast in real time in responseto the input of the user 2 via the tool.

The memory 130 may store various types of programs and data. Inparticular, the memory 130 may store a program for the creation ofaugmented reality content to be applied to an image, and may store aprogram for the provision of a live broadcast service.

A process that is performed by the controller 120 in order to createaugmented reality content to be applied to an image will be described indetail below with reference to the flowcharts of FIGS. 3 to 6.

FIGS. 3 to 6 are flowcharts illustrating methods for creating contentaccording to embodiments.

Referring to FIG. 3, at step 301, the controller 120 extracts a targetarea, to which augmented reality content will be applied, from an objectincluded in an image received from the camera 10. For example, thecontroller 120 may extract the face of a human included in an image asthe target area to which augmented reality content will be applied.Alternatively, the controller 120 may extract another body part (e.g., ahand, an arm, a leg, or the like) of the human included in the image asthe target area, or may extract a part or all of an object other than ahuman as the target area. In the present embodiment, it is assumed thatthe controller 120 extracts the face of the broadcaster 1 included inthe image as the target area.

Meanwhile, a part of an object in an image that is extracted as thetarget area by the controller 120 at step 301 may be set in advance, ormay be selected by the user 2 via the user terminal 20. For example, theuser 2 may acquire the right to apply augmented reality content to animage in return for a donation to the broadcaster 1. In this case, theuser 2 may be allowed to select the target area.

When the controller 120 attempts to extract the target area but fails atstep 301, the controller 120 may allow the user 2 to select anothertarget area, or may wait until the target area appears in the image. Forexample, when the controller 120 attempts to extract the face of a humanincluded in the image as the target area but the face of the human doesnot appear in the image, the controller 120 may request the user 2 toselect another body part other than the face of the human as the targetarea via the user terminal 20. Alternatively, the controller 120 maywait until the face of the human appears in the image, and may thenextract the target area.

The controller 120 may extract the target area by analyzing the image. Adetailed method by which the controller 120 extracts the target areawill be described below with reference to FIGS. 4 and 5.

FIG. 4 is a flowchart illustrating detailed steps included in step 301of FIG. 3. Referring to FIG. 4, the controller 120 may extracttwo-dimensional (2D) landmarks from an object included in the image atstep 401. In this case, the term “2D landmark” refers to a spotcorresponding to a specific region of an object within an image. Forexample, when the object is a human, the eyes, the nose, the mouth, theeyebrows, and the face lines may be extracted from the face of the humanas the 2D landmarks, or the hands, the feet, the arms, the legs, thewaist, and the shoulder may be extracted from the body as the 2Dlandmarks. The controller 120 may extract the 2D landmarks by analyzingthe RGB data of pixels that constitute the image.

At step 402, the controller 120 extracts a three-dimensional (3D) maskcorresponding to the target area by using the 2D landmarks extracted atstep 401. In this case, the term “3D mask” refers to the range of theimage that is a target to which augmented reality content will beapplied. When the face of a human included in the image is the targetarea as in the present embodiment, the surface of the face representedin a 3D space is the 3D mask.

A detailed process by which the controller 120 extracts a 3D maskcorresponding to a target area by using 2D landmarks will be describedwith reference to FIG. 5.

FIG. 5 is a flowchart illustrating detailed steps included in step 402of FIG. 4. Referring to FIG. 5, the controller 120 selects 2D landmarksincluded in the target area from among the 2D landmarks extracted fromthe object at step 501. For example, when the eyes, nose, mouth,eyebrows, face lines, hands, feet, arms, legs, waist, and shoulder of ahuman are extracted as the 2D landmarks at step 401, the controller 120selects the eyes, the nose, the mouth, the eyebrows, and the face lines,which are 2D landmarks included in the face, i.e., the target area, fromamong the former landmarks.

At step 502, the controller 120 extracts a 3D mask by calculating a 3Dmodel via the 2D landmarks selected at step 501. For example, thecontroller 120 extracts a 3D model corresponding to the face bycalculating a 3D mask using the locations of the eyes, the nose, themouth, the eyebrows, and the face lines, which are the selected 2Dlandmarks.

Referring back to FIG. 3, the controller 120 provides a template for thecreation of augmented reality content to be applied to the target areaat step 302. In other words, the controller 120 may provide a tool forthe creation of augmented reality content to the user 2 via the UI ofthe user terminal 20.

The user 2 may create the augmented reality content by a method ofinputting user content to a template that is displayed on the userterminal 20. In this case, the “user content” refers to a still image, amoving image, or the like that is directly input by the user 2 in orderto create augmented reality content. For example, the user 2 may draw astill image directly on the template, or may fetch a previously storedstill image, a previously stored moving image, or the like and place itat a desired location on the template.

In this case, the controller 120 may determine the form of the templateto be provided based on the selection of the user 2 or previously storedinformation about the user 2. This will be described in detail belowwith reference to FIG. 6.

FIG. 6 is a flowchart illustrating detailed steps included in step 302of FIG. 3. Referring to FIG. 6, the controller 120 determines whether ornot the user 2 has 3D modeling experience at step 601. In other words,the controller 120 determines whether or not the user 2 has thecapability to handle a 3D graphic tool. The information about the user 2may be stored in the memory 130 in advance, or may be checked in such away that the controller 120 inquires the information about the user 2from the user 2 via the user terminal 20 before the provision of thetemplate.

When as a result of the determination, it is determined that the user 2has 3D modeling experience, the process proceeds to step 602, at whichthe controller 120 provides a 3D mesh template corresponding to the 3Dmask to the user 2 via the user terminal 20. The user 2 may directlycreate augmented reality content through 3D modeling on the 3D meshtemplate.

In contrast, when as a result of the determination, it is determinedthat the user 2 does not have 3D modeling experience, the processproceeds to step 603, at which the controller 120 displays the 2D UVtemplate corresponding to the 3D mask on the user terminal 20. In thiscase, the 2D UV template is obtained by transforming the 3D mask into aUV coordinate system. The user 2 may easily create augmented realitycontent by inputting user content onto the 2D UV template.

Meanwhile, the controller 120 may provide a 3D preview along with the 2DUV template. When the user 2 inputs user content onto the 2D UVtemplate, a state in which the input user content has been applied tothe 3D mask may be displayed on the user terminal 20 as the 3D preview.

As described above, different types of templates are provided dependingon whether or not the user 2 has knowledge about 3D modeling, and thusan advantage arises in that even when the user 2 does not have thecapability to handle a 3D graphic tool, he or she may easily createcontent.

Although the embodiment in which the controller 120 determines the typeof template to be provided based on information about the user 2 isillustrated in FIG. 6, the user 2 may be allowed to select the type oftemplate via the user terminal 20 before the provision of the template.

An example in which when the user 2 inputs user content onto the 2D UVtemplate, a state in which the input user content has been applied tothe 3D mask is displayed on the 3D preview will be described below withreference to FIG. 7.

FIG. 7 shows screens that are displayed at the step of providing atemplate for the creation of content to a user in a method for creatingaugmented reality content according to an embodiment.

A 2D UV template 710 is displayed on the left side of FIG. 7, and a 3Dpreview 720 is displayed on the right side thereof. The 2D UV template710 is displayed on the user terminal 20. The user 2 may input usercontent onto the template via the user terminal 20, and may view apreview to which the result of the input is applied.

Referring to FIG. 7, when the user 2 inputs two images 711 and 712 ontothe 2D UV template 710 as user content, the two images 721 and 722 arealso displayed on the 3D preview 720, and thus a state in which actualaugmented reality content is displayed is shown.

The user 2 may easily create augmented reality content by a method ofattaching a 2D image onto the 2D UV template 710. It may be possible toimmediately view the 3D preview 720 to which an image is applied whileperforming the work of attaching the corresponding image, and thus anadvantage arises in that the user 2 may easily create augmented realitycontent.

Meanwhile, the controller 120 may provide a template of a type differentfrom that of the above-described example. According to an embodiment,the controller 120 may photograph a body part of the user 2corresponding to the target area via a camera provided in the userterminal 20, and may provide the photographed image as the template.

For example, when the target area to which augmented reality content isapplied is the face of the broadcaster 1 included in an image, the user2 photographs his or her own face by using the camera of the userterminal 20, and inputs user content onto his or her own face displayedon the screen of the user terminal 20. The controller 120 may transforminput user content into UV data, and may apply the resulting UV data tothe 3D mask of the target area.

Referring back to FIG. 3, at step 303, when the user 2 inputs usercontent via the template, the controller 120 may create augmentedreality content by using the input user content and the metadata of thetarget area. In greater detail, the controller 120 may create augmentedreality content by extracting the metadata of the target area and thencombining the metadata with data representing the user content input bythe user 2. In this case, the controller 120 may perform the calculationof the data representing the user content by using the metadata of thetarget area so that the created augmented reality content may bedisplayed on the location of the target area.

Meanwhile, when the 2D UV template has been provided at step 302, thecontroller 120 may create augmented reality content by performing UVmapping in a state in which input user content has been incorporatedinto the template and then performing combination with the metadata ofthe target area. Alternatively, when a photographed image of the user 2has been provided as the template at step 302, the controller 120 maycreate augmented reality content by incorporating UV data correspondingto user content input to the template into UV data corresponding to the3D mask of the target area, pertaining UV mapping, and pertainingcombination with the metadata of the target area.

The controller 120 may store the augmented reality content created atstep 303 in the memory 130, and may fetch and use the augmented realitycontent in response to a request from the user 2 in the future.

At step 304, the controller 120 may perform rendering so that theaugmented reality content created at step 303 is applied to the targetarea of the image. In greater detail, the controller 120 may performrendering by using data generated as a result of the combination of themetadata of the target area and the data representing the user contentinput to the template so that the augmented reality content isoverlapped and displayed on the target area.

In this case, the controller 120 may perform rendering so that theaugmented reality content is applied to the target area for a perioddetermined based on a request or payment (e.g., the payment of adonation) of the user 2.

The controller 120 may transmit the rendered image to the user terminal20. The controller 120 may provide live broadcast service bysimultaneously transmitting the rendered image to the user terminals ofother users that are not shown in FIG. 1.

FIG. 8 is a view showing an example in which augmented reality contentcreated by a method for creating augmented reality content according toan embodiment is applied to a live broadcast image. Referring to FIG. 8,augmented reality content 801 created by inputting user content, asshown in FIG. 7, is overlapped and displayed on a target area within animage 800. Furthermore, a phrase indicating a person who has made adonation in order to directly create the corresponding augmented realitycontent 801 and apply it to a live broadcast image and the amount ofdonation is displayed on the left side of the image 800.

As in the embodiments described above, the template for the creation ofaugmented reality content to be applied to a target area within an imageis provided to a user via the UI, and thus effects are achieved in thata user feels interesting because he or she may directly create desiredcontent and apply the content to an image broadcast in real time and inthat he or she actively participates in two-way communication throughbroadcasting.

Furthermore, the type of template to be provided is varied depending onthe selection or experience of a user, and thus a barrier to entry tothe creation of content is lowered, so that an effect is achieved inthat a user who lacks knowledge about 3D modeling may easily createaugmented reality content.

The term “unit” used herein means software or a hardware component suchas a field-programmable gate array (FPGA) or application-specificintegrated circuit (ASIC), and a “unit” performs a specific role.However, a “unit” is not limited to software or hardware. A “unit” maybe configured to be present in an addressable storage medium, and alsomay be configured to run one or more processors. Accordingly, as anexample, a “unit” includes components, such as software components,object-oriented software components, class components and taskcomponents, processes, functions, attributes, procedures, subroutines,segments in program code, drivers, firmware, microcode, circuits, data,a database, data structures, tables, arrays, and variables.

Components and a function provided in “unit(s)” may be coupled to asmaller number of components and “unit(s)” or divided into a largernumber of components and “unit(s).”

In addition, components and “unit(s)” may be implemented to run one ormore CPUs in a device or secure multimedia card.

The method for creating augmented reality according to at least one ofthe embodiments described in conjunction with FIGS. 3 to 6 may beimplemented in the form of a computer-readable medium that storesinstructions and data that can be executed by a computer. In this case,the instructions and the data may be stored in the form of program code,and may generate a predetermined program module and perform apredetermined operation when executed by a processor. Furthermore, thecomputer-readable medium may be any type of available medium that can beaccessed by a computer, and may include volatile, non-volatile,separable and non-separable media. Furthermore, the computer-readablemedium may be a computer storage medium. The computer storage medium mayinclude all volatile, non-volatile, separable and non-separable mediathat store information, such as computer-readable instructions, a datastructure, a program module, or other data, and that are implementedusing any method or technology. For example, the computer storage mediummay be a magnetic storage medium such as an HDD, an SSD, or the like, anoptical storage medium such as a CD, a DVD, a Blu-ray disk or the like,or memory included in a server that can be accessed over a network.

Furthermore, the method for creating augmented reality according to atleast one of the embodiments described in conjunction with FIGS. 3 to 6may be implemented as a computer program (or a computer program product)including computer-executable instructions. The computer programincludes programmable machine instructions that are processed by aprocessor, and may be implemented as a high-level programming language,an object-oriented programming language, an assembly language, a machinelanguage, or the like. Furthermore, the computer program may be storedin a tangible computer-readable storage medium (for example, memory, ahard disk, a magnetic/optical medium, a solid-state drive (SSD), or thelike).

Accordingly, the method for creating augmented reality according to atleast one of the embodiments described in conjunction with FIGS. 3 to 6may be implemented in such a manner that the above-described computerprogram is executed by a computing apparatus. The computing apparatusmay include at least some of a processor, memory, a storage device, ahigh-speed interface connected to memory and a high-speed expansionport, and a low-speed interface connected to a low-speed bus and astorage device. These individual components are connected using variousbuses, and may be mounted on a common motherboard or using anotherappropriate method.

In this case, the processor may process instructions within a computingapparatus. An example of the instructions is instructions that arestored in memory or a storage device in order to display graphicinformation for providing a Graphic User Interface (GUI) onto anexternal input/output device, such as a display connected to ahigh-speed interface. As another embodiment, a plurality of processorsand/or a plurality of buses may be appropriately used along with aplurality of pieces of memory. Furthermore, the processor may beimplemented as a chipset composed of chips including a plurality ofindependent analog and/or digital processors.

Furthermore, the memory stores information within the computing device.As an example, the memory may include a volatile memory unit or a set ofthe volatile memory units. As another example, the memory may include anon-volatile memory unit or a set of the non-volatile memory units.Furthermore, the memory may be another type of computer-readable medium,such as a magnetic or optical disk.

In addition, the storage device may provide a large storage space to thecomputing device. The storage device may be a computer-readable medium,or may be a configuration including such a computer-readable medium. Forexample, the storage device may also include devices within a storagearea network (SAN) or other elements, and may be a floppy disk device, ahard disk device, an optical disk device, a tape device, flash memory,or a similar semiconductor memory device or array.

The above-described embodiments are intended merely for illustrativepurposes. It will be understood that those having ordinary knowledge inthe art to which the present invention pertains can easily makemodifications and variations without changing the technical spirit andessential features of the present invention. Therefore, theabove-described embodiments are illustrative and are not limitative inall aspects. For example, each component described as being in a singleform may be practiced in a distributed form. In the same manner,components described as being in a distributed form may be practiced inan integrated form.

According to at least any one of the above-described embodiments, a usermay directly create augmented reality content and apply the augmentedreality content to an image while viewing the image in real time, andthus an effect is expected in that the user may feel more interest andsatisfaction, thus leading to active participation in live broadcasting.

Furthermore, an advantage arises in that a user having no 3D modelingexperience may easily create 3D augmented reality content by inputtinguser content via the 2D template.

The effects that may be acquired by the disclosed embodiments are notlimited to the above-described effects, and other effects that have notbeen described above will be clearly understood by those having ordinaryskill in the art, to which the disclosed embodiments pertain, from theforegoing description.

The scope of the present invention should be defined by the attachedclaims, rather than the detailed description. Furthermore, allmodifications and variations which can be derived from the meanings,scope and equivalents of the claims should be construed as fallingwithin the scope of the present invention.

What is claimed is:
 1. A method for creating augmented reality content,the method comprising: extracting a target area, to which augmentedreality content will be applied, from an object included in an image;providing a template for creation of the augmented reality content thatwill be applied to the extracted target area; when a user inputs usercontent via the template, creating the augmented reality content byusing the user content and metadata of the target area; and performingrendering so that the created augmented reality content is applied tothe target area of the image.
 2. The method of claim 1, whereinextracting the target area comprises: extracting two-dimensional (2D)landmarks from the object; and extracting a three-dimensional (3D) maskcorresponding to the target area by using the extracted 2D landmarks. 3.The method of claim 2, wherein extracting the 3D mask comprises:selecting 2D landmarks included in the target area from among the 2Dlandmarks extracted from the object; and extracting the 3D mask bycalculating a 3D model via the selected 2D landmarks.
 4. The method ofclaim 2, wherein providing the template comprises: providing a 2D UVtemplate corresponding to the 3D mask and a 3D preview; wherein the 2DUV template is obtained by transforming the 3D mask into a UV coordinatesystem; and wherein the 3D preview shows a state in which user contenthas been applied to the 3D mask when the user content is input onto the2D UV template.
 5. The method of claim 2, wherein providing the templatecomprises: providing any one of a 2D UV template and a 3D mesh template,or providing a photographed image of a body part of the usercorresponding to the target area as the template, based on a selectionof the user or previously stored information about the user.
 6. Themethod of claim 1, wherein creating the augmented reality contentcomprises: creating the augmented reality content by combining datarepresenting the input user content and metadata of the target area witheach other.
 7. The method of claim 6, wherein performing the renderingcomprises: performing rendering by using data obtained through thecombination so that the created augmented reality content is overlappedand displayed on the target area.
 8. The method of claim 1, whereinperforming the rendering comprises: performing rendering so that thecreated augmented reality content is overlapped and displayed on thetarget area of the image for a period that is determined based on arequest and payment of the user.
 9. A non-transitory computer-readablestorage medium having stored thereon a program that performs the methodset forth in claim
 1. 10. A computer program that is executed by anapparatus for creating augmented reality content and is stored in astorage medium to perform the method set forth in claim
 1. 11. Anapparatus for creating augmented reality content, the apparatuscomprising: a communication interface configured to communicate with anexternal device in order to transmit and receive data to and from theexternal device; a memory configured to store a program for creation ofaugmented reality content; and a controller configured to createaugmented reality content by executing the program; wherein thecontroller is further configured to extract a target area, to which theaugmented reality content will be applied, from an object included in animage received via the communication interface, to provide a templatefor creation of the augmented reality content, to be applied to theextracted target area, to a user terminal via the communicationinterface, to, when a user inputs user content via the template, createthe augmented reality content by using the user content and metadata ofthe target area, and to perform rendering so that the created augmentedreality content is applied to the target area of the image.
 12. Theapparatus of claim 11, wherein the controller is further configured toextract the target area by extracting two-dimensional (2D) landmarksfrom the object and extracting a three-dimensional (3D) maskcorresponding to the target area by using the extracted 2D landmarks.13. The apparatus of claim 12, wherein the controller is furtherconfigured to extract the 3D mask by selecting 2D landmarks included inthe target area from among the 2D landmarks extracted from the objectand extracting the 3D mask by calculating a 3D model via the selected 2Dlandmarks.
 14. The apparatus of claim 12, wherein the controller isfurther configured to provide the template to the user terminal byproviding a 2D UV template corresponding to the 3D mask and a 3Dpreview, the 2D UV template is obtained by transforming the 3D mask intoa UV coordinate system, and wherein the 3D preview shows a state inwhich user content has been applied to the 3D mask when the user contentis input onto the 2D UV template.
 15. The apparatus of claim 12, whereinthe controller is further configured to provide the template to the userterminal by providing any one of a 2D UV template and a 3D mesh templateor providing a photographed image of a body part of the usercorresponding to the target area as the template based on a selection ofthe user or previously stored information about the user.
 16. Theapparatus of claim 11, wherein the controller is further configured tocreate the augmented reality content by combining data representing theinput user content and metadata of the target area with each other. 17.The apparatus of claim 16, wherein the controller is further configuredto perform rendering so that the created augmented reality content isapplied to the target area of the image by performing rendering by usingdata obtained through the combination so that the created augmentedreality content is overlapped and displayed on the target area.
 18. Theapparatus of claim 11, wherein the controller is further configured toperform rendering so that the created augmented reality content isapplied to the target area of the image by performing rendering so thatthe created augmented reality content is overlapped and displayed on thetarget area of the image for a period that is determined based on arequest and payment of the user.